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Abstract 

The aim of this study was to develop a scale to determine the barriers affecting psychological help-seeking in college 
students. In line with this purpose, the validity and reliability of the scale were examined in five different studies. 
Exploratory factor analysis results suggested that the scale consists of five dimensions, labeled as fear of being 
stigmatized by society, trust in the mental health professional, difficulties in self-disclosure, perceived devaluation 
and lack of knowledge. The confirmatory factor analysis cross-validated the scale’s five-factor structure on a new 
college students’ sample. The convergent and divergent validity of the scale were also supported with theoretically 
and empirically expected correlation with self-concealment, self-stigma, and attitudes towards seeking psychological 
help. Moreover, the scale was found to have an adequate stability coefficient across three weeks and appropriate 
levels of internal consistency coefficient reliability for each of the subscales in two different samples. These initial 
findings show that the Barriers to Seeking Psychological Help Scale is a valid and reliable instrument to measure 
barriers related to psychological help-seeking for college students. Future studies can examine the validity and 
reliability of the scale on different samples and cultures, as well as predictive validity on college students’ decisions 
to seek counseling. 
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1. Introduction 

Previous research has shown that more than half of the mental disorders start before the age of 24 (Kessler et al., 
2005, 2007; Wang et al., 2005). According to the results of a nationwide study, mental disorders are also common in 
both adolescents and adults in Turkish society (Erol, K1I15, Ulusoy, Ke9eci, & §im§ek, 1998). This is concerning, 
because previous research also revealed that when these mental disorders are not treated, they can result in 
undesirable psychological, social, and economic problems including inadequate educational gains (Mojtabai et al., 
2015), low academic success (Keyes et al., 2012), dropping out of school (Melkevik, Nilsen, Evensen, Reneflot, & 
Mykletun, 2016), failures in social relationships (Idstad et al., 2015), substance abuse (Weitzman, 2004), and loss in 
the labor market (Rudolph & Eaton, 2015). Considering the prevalence of mental disorders worldwide, and the 
negative effects of these disorders if they are not treated, determining the barriers inhibiting people from utilizing 
mental health care services is of paramount importance for both maintaining community mental health and delivering 
effective mental health care services. 

Although mental disorders are prevalent and appear to be on the rise among college students (Hunt & Eisenberg, 
2010), studies have consistently shown that college students underutilize mental health services for their academic 
and emotional problems (Topkaya, 2014a, 2014b, 2015a; Vogel, Gentile, & Kaplan, 2008; Vogel, Wade, & Hackler, 
2008; Wang et al., 2005). Given that counseling has numerous benefits and not seeking counseling, on the other hand, 
brings many adverse consequences (Eisenberg, Golberstein, & Gollust, 2007). The early diagnosis and treatment of 
mental problems in emerging adults, a period that roughly spans the ages of 17 to 25 in Western countries, is of 
critical importance. In order to effectively direct people to receive mental health care services, the barriers inhibiting 
them from seeking help need to be determined. Thus, in recent years, some research has been conducted to determine 
the barriers influencing people’s willingness to seek psychological help when they are faced with intrapersonal and 
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interpersonal problems (Kuhl, Jarkon-Horlick, & Morrissey, 1997; Setiawan, 2006). Barriers related to psychological 
help seeking are defined as a general unwillingness to receive psychological help or factors hindering an individual’s 
psychological help seeking behavior (Kuhl et al., 1997). Kuhl et al. (1997) revealed that the most important factor 
inhibiting seeking psychological help is perceiving oneself, family, or friends as adequate in solving problems. In a 
study with high school students, Wilson and Deane (2012) examined the relationship between the intention to seek 
psychological help for personal, emotional, and suicidal problems, and the barriers to seeking psychological help in 
adolescents. Researchers found that students emphasized autonomy as the most influential inhibiting factor for 
themselves. Students believed that they were competent enough to solve their own problems without any 
professional help. Other important barriers identified in this study were feeling ashamed and restrained, fear of being 
misunderstood, being forced to do things they did not want to do, and issues surrounding time and money. In another 
study concerning the barriers related to seeking psychological help. Stanhope (2002) found that male students 
perceived more barriers than female students. Also, the barriers for students were categorized as the fear of being 
stigmatized, arranging an appropriate time for counseling, privacy concerns, doubts about the usefulness of the 
mental health professional, self-awareness, and feeling oneself competent. Lastly, Calloway (2008) concluded that 
while the barriers to receiving psychological help were the fear of being stigmatized, fears concerning treatment, and 
issues about self-disclosure and suicidal tendencies in undergraduate students, the most significant barriers was 
found to be the fear of being negatively judged by others. 

The majority of previous studies have attempted to identify the factors related to the attitudes towards seeking 
psychological help in Turkey. These studies most commonly investigated the sociodemographic correlates (e.g., 
gender, class level, perceived socio-economic status, parental education level, past counseling experience) in 
attitudes towards seeking psychological help (Keklik, 2009; Koydemir-Ozden, 2010). Other researchers investigated 
the problems undergraduate students encounter and the psychological resources they use to cope with them (Ktling & 
Granello, 2003), as well as undergraduate students’ willingness to seek psychological help and the associated factors 
(Erkan, Ozbay, Cihangir-Cankaya, & Terzi, 2012). More recent studies have focused on the factors affecting college 
students' and adults’ willingness to seek psychological help (Topkaya, 2014a, 2014b, 2015a, 2015b). These studies 
revealed some facilitators and barriers related to psychological help-seeking. Such that being female, past counseling 
experience, positive attitudes towards help-seeking, the availability of psychological services, the belief in the 
benefits of psychological services, trusting in the mental health professional, and receiving psycological help free of 
charge is more likely to be facilitating factors in help-seeking process in Turkish people. On the other hand, 
self-stigma, public stigma, unwillingness to share problems with an unfamiliar persons, and self reliance is more 
likely to be barriers in help-seeking process. 

However, to date, we are not aware of any study focusing on barriers related to seeking psychological help in Turkey, 
except for a quantitative research study with undergraduate students (Koydemir, Erel, Yumurtacr, & §ahin, 2010). 
Specifically, Koydemir et al. (2010) examined students’ opinions concerning the barriers related to seeking 
psychological help, university adaptation, students’ needs, their psychological resources used, and their attitudes 
towards seeking psychological help. Following the interviews conducted in the context of study, the most commonly 
mentioned barriers to seeking psychological help was that students did not believe that they had any problem they 
could not solve on their own. In other words, they thought that they did not have a problem that required them to 
seek help from a professional. However, these people stated that they would consider seeking psychological help 
from a mental health professional in troubled times. The belief in being able to solve problems on their own, seeking 
help from close friends, and reluctant attitudes towards self-disclosure to a mental health professional were among 
the other reported barriers in the study. 

Taking into account the above studies that have begun to identify potential barriers to seeking treatment, the barriers 
to seeking psychological help can be grouped under three categories; personal barriers (e.g., personal features and 
situations), socio-cultural barriers (e.g., cultural values or characteristics of a specific community), and barriers 
related to institutions delivering help (e.g., psychological counseling services, psychological counselors, and other 
administrative issues) (Setiawan, 2006). Recent research demonstrated that there is a negative relationship between 
willingness to seek psychological help and perceived barriers (Setiawan, 2006). In other words, as the number of 
barriers people perceive increases, their willingness to seek psychological help tend to decrease. Considering the 
studies related to seeking psychological help in Turkey, there has not been any research focusing directly on the 
barriers to seeking psychological help. Additionally, there have not been any instruments developed that can be used 
to determine the barriers college students perceive, except for Mansfield, Addis, and Courtenay (2005),which 
focused on revealing male college students’ barriers related to seeking help about health problems. Two obvious 
main limitations exist in this instrument. Firstly, this scale developed in American male college students to determine 
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their help-seeking barriers in health related issues. Barriers related to psychological help may differ from other health 
related problems at least in magnitude and severity. Studies have shown that regardless of the specific psychiatric 
diagnosis or level of disability, people tend to stigmatize people with mental illness more severely than those people 
with other health problems (Corrigan et al., 2000; Weiner, Perry, & Magnusson, 1988). Secondly, male sample also 
makes it difficult to generalize to barriers other samples such as females. One of the most consistent results in the 
diverse area of help seeking, including academic work, counseling, physical and mental health is that females more 
likely to seek help to a greater degree than do male (Nam et ah, 2010). Thus, identifying common barriers to 
help-seeking process is crucial for developing nuanced interventions for both sexes. The aim of this study is to 
develop a scale to determine the barriers affecting undergraduate students seeking psychological help and to examine 
its validity and reliability in college students. Investigating these barriers can contribute to revealing and eliminating 
the factors that prevent undergraduate students from receiving mental health care services. Also, determining these 
barriers is of importance in terms of planning appropriate interventions to support students’ mental health and 
well-being, and enhance mental health care services (Stanhope, 2002). 

2. Method 

2.1 Participants 

The Barriers to Seeking Psychological Help Scale (BSPHS) was developed through the five different studies. 
Participants were recruited using the convenience sampling method except for fourth study. The participants of first 
study was used to generate the item pool related to the scale and construct a draft scale; the participants of second 
study was employed to determine the number of factors to retain in the draft scale and exploratory factor analysis. 
The participants of third study was used for confirmatory factor analysis in order to determine whether the factor 
structure of the scale was similar in a different sample. The participants of fourth study consisted of randomly 
recruited participants from the second and third sample used to examine the BSPHS’s convergent and divergent 
validity. Lastly, the fifth sample was employed to determine the BSPHS’s test-retest reliability across three weeks. 

The participants of first study consisted of 353 students from various faculties and departments in a university in the 
Aegean region of Turkey. There were 171 (49%) male and 182 (51%) female participants. Of the 353 participants, 93 
(26%) were in their first university year; 100 (28%) were in their second university year; 97 (27%) were in their third 
university year; and lastly, 68 (19%) were their fourth university year. 

The participants of second study similarly consisted of 205 students from various faculties and departments in a 
university in the Aegean region of Turkey. Of the 205 students, 77 (38%) were male and 128 (62%) were female. The 
age of the students ranged from 18 to 27, and the mean age was 21.00 ( S.D.: 2.00). Most of the participants were first 
year students (38%), followed by second year students (32%), third year students (22%), and fourth year students 
(8%). There were 45 (22%) students who had previously sought psychological help. 

The participants of third study consisted of 222 students from various faculties and departments in a university in the 
Aegean region of Turkey. There were 87 (39%) male and 135 (61%) female participants. The participants’ age 
ranged from 18 to 39, and the average age was 21.32 ( S.D.: 2.26). Of these participants, 72 (32%) were in their 
freshmen year, 62 (28%) were in sophomore year, 55 (25%) were in junior year, and 33 (15%) were in senior year. 
There were 54 (24%) students who had received psychological help. 

The participants of fourth study consisted of randomly selected participants from the second and third study. We give 
information students in the second and third study about that some questionnaires also included scales for another 
study that aims to determine the correlates of barriers to seeking psychological help before distributed the 
questionnaire. Students who voluntarily to answer these questionnaires composed of fourth sample. There were 245 
(92 males and 153 females) students who ranged in age from 18 to 39, and the average age was 21.47 ( S.D.: 2.35). 
The majority of these students were in their sophomore year (n=77, 31%). 

Lastly, the participants of fifth study composed of 91 students from the education faculty from a university in the 
Central Black Sea Region of Turkey. The age range of participants was from 18 to 25, and the average age was 19.97 
( S.D.: 1.64). Most of the students were in their freshmen year (7; =48, 53%), and had not sought any psychological 
help before (« = 82, 90%). 

2.2 Development Process of Barriers to Seeking Psychological Help Scale (BSPHS) 

Initially, the existing literature was reviewed so as to develop content domain for the BSPHS. Specifically, we 
reviewed previous studies that empirically examined inhibiting factors in psychological help seeking in student or 
adult samples (e.g., Calloway, 2008; Setiawan, 2006; Stanhope, 2002). Moreover, two open-ended questions were 
asked of students to determine encouraging and inhibiting factors in the help-seeking decision. These questions were 
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“What are the ‘facilitative/encouraging’ factors that help you receive psychological help?” and “What are the 
‘inhibiting/hindering’ factors that stop you from receiving psychological help?” The students were asked to answers 
these questions based on their own opinions and previous experiences. Using students' answers to the questions as 
well as items we constructed based on the literature review, an item pool was generated including 35 items. After 
constructing the item pool, a draft scale was prepared accordingly and sent to seven experts. There were three experts 
in Guidance and Psychological Counseling, two experts in the Turkish Language and two experts in Measurement 
and Evaluation. After the definition of barriers related to psychological help seeking (Kuhl et ah, 1997), experts rated 
each draft item’s clarity and content validity on a five-point scale ranging from Absolutely unsuitable (1) to 
Absolutely suitable (5). We calculated average expert ratings for each draft item based on clarity and content validity 
scores separately and then excluded them if the mean expert rating was below three for either the clarity or content 
validity score. As a result of this process, we removed 18 items from the draft scale, and finally, the 17-item BSPHS 
was constructed. 

2.3 Other Scales 

In order to determine convergent and divergent validity estimates of BSPHS in fourth sample, we used The Self 
Concealment Scale, The Self-Stigma of Seeking Help Scale, and The Attitudes towards Seeking Psychological Help 
Scale-Revised. Psychometric properties of these scales is discussed in further detail below. 

Self Concealment. The Self Concealment Scale (SCS) was used to measure the self-concealment levels of college 
students. The SCS was developed by Larson and Chastain (1990) and adapted to Turkish culture by Terzi, Giingor, 
and Erdayi (2010).This one-dimensional scale measures people's tendencies to actively conceal personal information 
about secrets, negative and distressing life experiences, and thoughts about themselves. The SCS consists of 10 items 
and participants are asked to indicate to what extent they agree or disagree with each item on a five-point scale 
ranging from Strongly Disagree (1) to Strongly Agree (5). The scale does not include reverse coded items, and the 
scores of the scale range from 10 to 50. Higher scores indicate higher levels of self-concealment. The internal 
consistency coefficient of the scale for this study was (a) .78. An example item from the SCS is “I have an important 
secret that I haven’t shared with anyone.” 

Self-Stigma. The Self-Stigma of Seeking Help Scale (SSOSHS) is used to measure self-stigma associated with 
seeking psychological help. The SSOSHS was developed by Vogel, Wade, and Haake (2006) and adapted to the 
Turkish language by Topkaya (2011). Both the original and adapted scale is a one-dimensional structure consisting of 
10 items and the participants respond to each item on a five-point Likert scale ranging from Strongly Disagree (1) to 
Strongly Agree (5). Items 2, 4, 5, 7, and 9 are reverse scored and then all items are summed for a total score. The total 
scores that can be obtained from the scale range from 10 to 50. However, the SSOSHS’s tenth item was not included 
in this study for statistical analysis because it exhibited very low item-total correlation. Thus, the scores of the 
SSOSHS range between 9 and 45 for this study. Higher scores indicate greater self-stigma associated with receiving 
psychological help from a mental health professional (Topkaya, 2011). The internal consistency coefficient 
(Cronbach’s Alpha) of the scale for this study was .70. An example item from the SSOSHS is “Seeking 
psychological help would make me feel less intelligent.” 

Attitudes towards Seeking Psychological Help. The Attitude towards Seeking Psychological Help Scale-Revised 
(ATSPHS-R) is used to measure people’s attitudes towards seeking psychological help. This scale was developed by 
Tiirkum (2001) using the previously validated Attitude towards Seeking Psychological Help Scale in the context of 
Turkish culture. The ATSPHS-R is comprised of 18 items and each item is rated on a five-point scale ranging from 
Strongly Disagree (1) to Strongly Agree (5). The total scores that can be obtained from the scale range from 18 to 90. 
High scores indicate more positive attitudes towards seeking psychological help. The test-retest reliability across the 
10 weeks was measured at .77 and the Cronbach Alpha internal consistency of the scale in college students was .90 
(Tiirkum, 2001). The Cronbach Alpha internal consistency coefficient of the scale for this study was .87. An example 
item from the ATSPHS-R is “When a close friend of mine asked me opinions about his/her mental health difficulties, 
I can recommend him/her to seek psychological help.” 

2.4 Procedure 

Approval to conduct the study was obtained from Human Subjects Ethics Committee of related universities prior to 
study commencement. Following the ethics committee approvals, the participants were asked to complete the 
questionnaires, which were delivered to the participants by the first or third researcher at the beginning or end of 
their courses during regular class hours. Ethical considerations of the study clearly assured before administration of 
questionnaires. The students were informed about voluntary participation, confidentiality, anonymity, and that they 
could withdraw at any step of the study without any sanction. Students did not receive any incentives in return for 
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their participation in the study and all students participated voluntarily. All participants in the studies completed the 
BSPHS. The participants in the fourth sample also completed the SCS, SSOSHS, and ATSPHS-R. The BSPHS was 
administered to participants in the fifth sample again after three weeks for calculating test-retest reliability. 
Participants completed the questionnaires in about 20 to 30 minutes depending on the studied sample. 

2.5 Statistical Analyses 

The statistical analyses were performed with SPSS 23 e , Factor 10.3 (Lorenzo-Seva & Ferrando, 2006, 2013), and 
Mplus 7.4 (Muthen & Muthen, 1998-2014) statistical computer programs. Among all studies, 43 participants who did 
not answer most parts of the questionnaire were excluded from the data set. In the fifth sample, which was conducted 
to determine test-retest reliability, some of the participants could not answer all of the items. Specifically, there were 
five participants who could not answer only one item, and two participants who could not answer two items in the 
first administration. In the second administration, one participant did not answer three items, one participant did not 
answer two items, and three participants did not answer one item. For this limited number of participants with 
missing values, we replaced these missing values using the expectation-maximization missing data estimation 
method. Univariate and multivariate outliers were checked, and eight multivariate outliers in the second sample and 
six multivariate outliers in the third sample were excluded from the data set (Hair, Black, Babin, & Anderson, 2014). 
In order to determine underlying factor structure of the BSPHS, an exploratory factor analysis was conducted with 
the second study data. Before conducting the exploratory factor analysis, as suggested by measurement and 
evaluation experts (Costello & Osborne, 2005; Fabrigar, Wegener, MacCallum, & Strahan, 1999; Izquierdo, Olea, & 
Abad, 2014), several tests were performed in order to determine whether the correlation matrix of the data was 
suitable for exploratory factor analysis and to ascertain the number of factors to retain. 

Firstly, whether there was an adequate level of correlation among scale items was examined through Bartlett’s test of 
sphericity. Also, whether the sample size was sufficient to conduct factor analysis was analyzed with the 
Kaiser-Meyer-Olkin sampling adequacy coefficient. The Barlett Tests for Sphericity test whether the correlation 
matrix of the data is an identity matrix. To be able to conduct a factor analysis, the Barlett Tests for Sphericity results 
should be significant, and the Kaiser-Meyer-Olkin (KMO) sample adequacy coefficient should be .60 or above 
(Kaiser, 1974). In order to determine the number of factors to retain in the scale, multiple factor retention decision 
criteria, including eigenvalue greater than one, scree-test, parallel analysis, Velicer’s Map Test, and Hull method were 
used. Additionally, the coherence among scale items and the interpretability of the factors were other criteria. Also, 
another criteria was that there should be at least three items in each subscale (Hair et ah, 2014). The eigenvalue 
greater than one rule, also known as in the literature as the Kaiser-Gutman Rule, posits that factors with an 
eigenvalue higher than one significantly contribute to the explained variance rate and the factors with an eigenvalue 
less than one contribute to explained variance less or trivial amount. Thus, the eigenvalue greater than one rule 
recommends to retain factors with eigenvalues higher than one. The Scree test aims to determine the number of 
factors to retain by visually analyzing eigenvalues in a scree-plot graphic. When the eigenvalues occur in a 
broken-stick pattern, researchers can find the true number of factors to retain. Parallel analysis produces random 
correlation matrixes with the same number of items and sample size in the scale and calculates the average 
eigenvalues using this virtually obtained correlation matrix, then the researchers compare these virtually obtained 
mean eigenvalues with their actual sample eigenvalues. The last sample eigenvalue higher than the virtually obtained 
eigenvalue determines the factor number in the scale. Velicer’s Map test states that the common factors in a scale are 
associated with latent factors, and these common factors show partial correlation with latent factors until the last true 
factor in the scale. In other words, Velicer’s Map test stops to extract the factors when partial correlation reaches the 
minimum level. Velicer’s Map test uses principal component analysis to calculate partial correlations. Lastly, the 
Hull method aims to find a model with the optimal balance between model and number of factors using goodness of 
fit indexes (e.g.. Comparative fit index (CFI), Root Mean Square Error of Approximation (RMSEA)) (Cattell, 1966; 
Horn, 1965; Lorenzo-Seva & Ferrando, 2006; Lorenzo-Seva, Timmerman, & Kiers, 2011; Velicer, 1976). 

After determining the number of factors in the scale, exploratory factor analysis was performed with Principal Axis 
Factoring and Promax rotation (k= 4). The reason for using Principal Axis Factoring is that this factoring approach 
does not require that scale items meet specific assumptions, such as multivariate normal distribution (Fabrigar et al., 
1999). The reason for using oblique Promax rotation is that we expect correlations among obtained factors (Costello 
& Osborne, 2005) and there was no theoretical assumption demonstrating that the obtained factors would be 
independent from each other. At the same time, in order to determine which item belonged to which factor, we used 
item factor loading values of .30 or higher (Field, 2013), and there was to be a difference of at least .10 or higher 
between corresponding salient item factor loadings and other factors. 
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A confirmatory factor analysis was conducted with the third sample to cross-validate the underlying factor structure 
of the BSPHS in a new sample. The robust maximum likelihood method (MLR) was used as an estimation method in 
the confirmatory factor analysis. The MLR method corrects standard errors when data violate the multivariate normal 
distribution assumption. In confirmatory factor analysis, the hypothesized model’s fit to data is examined through a 
number of goodness of fit statistics, including adjusted Chi-Square (j/ 2 /df), RMSEA with 90% confidence interval, 
CFI, Tucker-Lewis Index (TLI), Standardized Root Mean Square Residual (SRMR) (Hu & Bentler, 1999; Marsh, 
Balia, & McDonald, 1988). Values less than 5 but more than 3 are acceptable for y 2 /df, and values less than 3 
indicate a perfect fit. For RMSEA, the values between .08 and .06 show good fit, while values less than .06 indicate 
excellent fit. For CFI and TLI values, .90 and .95 indicate acceptable and excellent fit. Lastly, SRMR values of .08 
and less indicate good fit (Hu & Bentler, 1999; Marsh et ah, 1988). 

Lastly, in order to determine convergent and divergent validity, BSPHS subscales’ correlation with other theoretically 
relevant scales were examined with Pearson correlation analysis in the fourth sample. Similarly, the Pearson 
correlation analysis was also used with the fifth sample in order to determine the stability coefficient of BSPHS 
subscales, while Cronbach Alpha reliability was used to estimate the internal consistency of the subscales and total 
scale. 

3. Results 

3.1 Study 2: The Determination of Factor Number and Exploratory Factor Analysis 

Bartlett’s Test of Sphericity found that the correlation matrix was not an identity matrix (X 2 fi 36 r= 1417.30, p <.001), 
and the Kaiser Meyer Olkin sampling adequacy coefficient was .86, well above the suggested minimum value for 
conducting a factor analysis (Kaiser, 1974). The item level sampling adequacy coefficient (anti image correlation) 
also ranged from .69 to .92. These findings suggested that the correlation matrix of the scale items was appropriate 
for conducting factor analysis. 

The eigenvalue greater than one rule found that the correlation matrix of the scale items consisted of five factors with 
an eigenvalue higher than unity. The scree plot results also showed a broken-stick pattern in factors two and five. 
While Horn’s (1965) parallel analysis suggested the extraction of the two factors, Velicer’s MAP test recommended 
the extraction of only one factor. Similarly, Hull’s method also suggested retaining one factor. Since multiple factor 
retention decision criteria suggested extracting different numbers of factors, we performed three separate factor 
analyses using one-, two-, and five-factor forced solutions with Principal Axis Factoring. When we extracted two or 
five factors, as previously specified the oblique Promax rotation (k= 4) was used. It was found that forced one-factor 
and two-factor solutions did not reveal an interpretable factor structure. However, a forced five-factor solution 
yielded an interpretable factor structure and good coherence between items in each factor. 

Table 1 shows a forced five-factor solution using Principal Axis Factoring and Promax rotation. As seen in Table 1, 
Factor I consists of four items and factor loadings ranged from .74 to .94, which explains 34.24% of the total 
variance. When item contents were scrutinized, this factor indicated the people’s fears related to being stigmatized by 
society when they receive psychological help. Thus, this factor is named fear of being stigmatized by society. Factor 
II similarly includes four items with factor loadings of .31 to .81. This factor explains 10.30% of the total variance. 
Since this factor includes items related to the difficulties people feel trusting mental health professionals, this factor 
is called trust in the mental health professional. Factor III consists of three items with factor loadings that ranged 
from .64 to .87. This dimension reflects the items related to people’s self-disclosure to a mental health professional, 
thus it was called difficulties in self-disclosure. Factor IV includes three items with factor loadings of .40 to .86. This 
factor consists of items that evaluate people’s negative thoughts about themselves when they receive psychological 
help. Thus, this factor is called perceived devaluation. And lastly. Factor V includes three items with factor loadings 
ranging from .33 to .95. This factor contains the items regarding people’s lack of knowledge about seeking 
psychological help, so it is called lack of knowledge. The BSPHS is presented in Appendix A. 

3.2 Study 3: Confirmatory Factor Analysis 

A confirmatory factor analysis was conducted to test whether the BSPHS would have a similar factor structure in a 
new sample. Based on current factor analysis and different factor retention decision criteria results, three possible 
competing models were tested in the confirmatory factor analysis in the third study: One-factor model (Model 1), 
two correlated factor model (Model 2), and suggested five correlated factor model (Model 3). The two-factor model 
estimated was based on exploratory factor analysis results. Table 2 shows the goodness of fit indexes of competing 
models based on confirmatory factor analysis. 
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Table 1. Exploratory Factor Analysis Results of BSPHS 


Factor Loadings 


Factors 

I 

II 

III 

IV 

V 

h 2 

Item9 

.94 





.81 

Item2 

.83 





.67 

Item6 

.81 





.77 

Iteml4 

.74 





.64 

Item 16 


.81 




.63 

Item 17 


.80 




.60 

Item 12 


.50 




.35 

Item4 


.31 




.29 

Item5 



.87 



.73 

Item8 



.70 



.52 

Iteml 



.64 



.50 

Item 13 




.86 


.77 

Iteml 1 




.68 


.49 

Item 15 




.40 


.31 

Item 10 





.95 

.88 

Item7 





.44 

.21 

Item 3 





.33 

.19 

Eigenvalue 

5.82 

1.74 

1.42 

1.20 

1.14 


Explained Variance (%) 

34.24 

10.30 

8.33 

7.08 

6.71 



Note. h 2 = communality. Absolute factor loading values less than .30 were discarded for the sake of clarity. 
Table 2. Goodness of Fit Indexes of Competing Models 


Model 

x 2 

df 

/>(/' 

CFI 

TLI 

RMSEA 

90 % Cl 

SRMR 

One Factor 

408.026 

119 

3.429 

.719 

.679 

.105 

.094-. 116 

.085 

Two BSPHS Factors 

368.697 

117 

3.145 

.756 

.716 

.098 

.087-.110 

.087 

Five BSPHS Factors 

138.343 

109 

1.269 

.972 

.964 

.035 

.012-.051 

.053 


Note. The two-factor model estimated is based on exploratory factor analysis results. Correlations between latent factors in 
two-factor model and five-factors models were allowed. Latent factor variances were fixed to one for model identification 
purposes. C.I.: Confidence Interval. 


As seen in Table 2, the one-factor model and two-factor model showed poor fit to data. However, the five correlated 
factor model showed excellent fit to data. These findings confirmed the factor structure of the scale which was 
obtained from exploratory factor analysis in a new sample. Table 3 demonstrates the standardized item factor 
loadings, t-values, and squared multiple correlations. As seen in Table 3, all factor loadings and t-values were large 
and statistically significant. 
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Table 3. 

Results of Confirmatory Factor Analysis 




Standardized Factor Loadings 




I II III IV V 

t 

R 2 

Factors 



Item9 

.81(.05) 

16.64 

.66 

Item2 

.90003) 

36.51 

.81 

Item6 

.91 (.02) 

39.38 

.82 

Iteml4 

.71 (.06) 

12.29 

.51 

Iteml6 

.77005) 

15.12 

.60 

Iteml7 

.72005) 

14.16 

.52 

Iteml2 

.72006) 

12.50 

.51 

Item4 

.49(.06) 

8.34 

.24 

Item5 

.81005) 

18.17 

.66 

Item8 

.77004) 

17.45 

.59 

Iteml 

.66007) 

10.00 

.43 

Iteml3 

.81005) 

16.73 

.65 

Iteml 1 

.79008) 

10.07 

.62 

Iteml 5 

,57(.07) 

8.67 

.33 

Iteml 0 

.78007) 

11.89 

.60 

Item7 

.75005) 

13.98 

.56 

Item 3 

.68007) 

9.49 

.47 


Note. Numbers in parenthesis are standard errors of factor loadings. All t-values and R 2 values were significant at p<.001. 
3.4 Study 4: Convergent and Divergent Validity 


In order to determine convergent and divergent validity of the BSPHS, the relationship between the subscale scores 
of the BSPHS and attitudes towards seeking psychological help, self-concealment, and self-stigma scores were 
examined using Pearson correlation analysis. It was expected that, as an individual’s barriers to seeking 
psychological help increased, attitudes towards seeking psychological help scores would decrease and 
self-concealment and self-stigma scores would increase. Table 4 presents the correlations among the study variables. 

As shown in Table 4, the subscale scores of the BSPHS are positively correlated with each other in the small to 
medium range. Moreover, the subscale scores of the BSPHS are positively correlated with self-stigma and 
self-concealment scores, evidencing convergent validity of the BSPHS, whereas the subscale scores of the BSPHS 
are negatively correlated with attitudes towards psychological help-seeking scores, evidencing divergent validity of 
the BSPHS scores. In other words, as individuals’ barriers related to fear of being stigmatized by society, trust in the 
mental health professional, difficulties in self-disclosure, perceived devaluation, and lack of knowledge increase, the 
self-concealment and self-stigma levels tend to increase and they tend to hold more negative attitudes towards 
seeking psychological help. These findings support convergent and divergent validity of the BSPHS. 
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Table 4. Correlations among Variables 


1 

2 

3 

4 

5 

6 

7 

8 

1. Fear of being stigmatized by society 









2. Trust in the mental health professional 

.43 








3. Difficulties in self-disclosure 

.52 

.44 







4. Perceived devaluation 

.52 

.50 

.51 






5. Lack of knowledge 

.41 

.37 

.36 

.29 





6 . Self-stigma 

.29 

.32 

.22 

.24 

.22 




7. Self-concealment 

.28 

.24 

.25 

.21 

.13* 

.33 



8 . Attitude towards seekingpsychological help 

-.23 

-.31 

-.22 

-.28 

-.28 

-.61 

-.27 


Mean 

7.71 

10.07 

7.77 

6.52 

6.57 

21.74 

24.57 

65.92 

S.D. 

3.30 

3.15 

2.69 

2.44 

2.41 

6.07 

7.19 

11.51 


Note. p<. 05*. All other correlations were significant at p<.0 1. 
3.5 Study 2, 3, and 5: Reliability of the BSPHS 


The reliability of the BSPHS was examined through test-retest and Cronbach Alpha internal consistency coefficient. 
The test-retest reliability study results obtained though the administration of the scale twice within a three-week 
interval produced.66 for fear of being stigmatized by society, .69 for the trust in the mental health professional, .71 
for difficulties in self-disclosure, .56 for perceived devaluation, and .66 for the lack of knowledge subscales. Table 5 
demonstrates the Cronbach Alpha internal consistency coefficients obtained from the second and third samples. 

Table 5. Cronbach Alpha Internal Consistency Coefficients across Different Samples 

Study 2 Study 3 


Fear of being stigmatized by society 

.91 

.90 

Trust in the mental health professional 

.73 

.77 

Difficulties in self-disclosure 

.77 

.78 

Perceived devaluation 

.71 

.75 

Lack of knowledge 

.58 

.78 

Total scale 

.87 

.91 


As seen in Table 5, the Cronbach Alpha internal consistency coefficient of the scale in the second sample ranged 
from .58 (lack of knowledge) to .91 (fear of being stigmatized by society), while the Cronbach Alpha internal 
consistency coefficient of the scale in the third sample ranged from .75 (perceived devaluation) to .90 (fear of being 
stigmatized by society). The Cronbach Alpha internal consistency coefficient of the total scale was .87 for the second 
sample and .91 for the third sample. 

4. Discussion 

The college years is a period when most students have to cope with new challenges and new experiences. They may 
face some personal, social, and academic problems, which may require them to seek psychological help in this 
period (Gtineri, Aydin, & Skovholt, 2003; Koydemir et ah, 2010; Setiawan, 2006). However, studies have 
consistently shown that their willingness to seek psychological help to cope with these problems is often not in 
accordance with the intensity of their problems (Mansfield et ah, 2005; Setiawan, 2006). Moreover, it is obvious that 
there are various factors influencing whether university students seek psychological help from a mental health 
professional (Kakhnovets, 2011; Mansfield et ah, 2005; Setiawan, 2006). Unfortunately, there is no instrument to 
determine these factors and how they influence individuals seeking psychological help, or which factors are more 
important than others in this process. Thus, the purpose of this study was to develop an instrument to evaluate the 
barriers affecting university students seeking psychological help. 

Consistent with this purpose, firstly, the possible factors influencing the intentions of university students to seek 
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psychological help were determined through an extensive literature review and the use of two open-ended questions 
to uncover facilitating and inhibiting factors. After the content analysis and literature review, a 35-item draft form 
was prepared. It was then presented to experts for review. Based the results, 18 items were discarded from the draft 
and the final form of the scale was created with 17 items. 

Exploratory factor analysis results suggested that the scale should consist of five dimensions: fear of being 
stigmatized by society, trust in the mental health professional, difficulties in self-disclosure, perceived devaluation, 
and lack of knowledge. These dimensions explained 66.65% of the total variance. The explained total variance rate in 
the literature for multidimensional instruments accounting for 50% or more is accepted as adequate for scales 
(Beavers et al., 2013). In this respect, the BSPHS is an adequate scale to measure factors that inhibit college students 
from seeking psychological help. In the confirmatory factor analysis with a different sample, three competing models 
were tested. The results suggested that the goodness of fit index of one-factor and two-factor models did not fit the 
sample data, but the goodness of fit index of the five-factor model was perfect. Thus, the factor structure obtained 
from the exploratory factor analysis was confirmed in a new sample. The results of this study lend further support to 
previous studies showing that stigma, trust and confidentiality, lack of knowledge, difficulties to express emotions is 
a barrier in psychological help-seeking process (Kakhnovets, 2011; Kuhl et al., 1997; Setiawan, 2006). 

The convergent and divergent validity estimates of the scale were examined by correlating relevant psychological 
constructs, including attitudes towards seeking psychological help, self-concealment, and self-stigma associated with 
receiving psychological help. Previous research has shown that there was a negative relationship between self-stigma 
and attitudes towards seeking psychological help, between public stigma and attitude towards seeking psychological 
help, and between self-concealment and attitudes towards seeking psychological help (Li, Dorstyn, & Denson, 2014; 
Nam et al., 2013). Thus, it was expected that there would be a positive relationship between barriers to seeking 
psychological help subscales and self-stigma and self-concealment, while there would be a negative relationship 
between the attitudes towards seeking psychological help. In the study, it was observed that the subscales of the 
BSPHS were weakly but positively correlated with self-stigma and self-concealment. Moreover, the subscales of the 
BSPHS were also weakly but negatively associated with the attitudes towards seeking psychological help. These 
findings support convergent and divergent validity of the BSPHS. Lastly, the reliability of the scale was examined 
using test-retest reliability and Cronbach Alpha internal consistency coefficient. Reliability of the BSPHS was 
assessed via test-retest correlations using a three-week time interval that revealed that the stability coefficient was in 
the acceptable to good range(between .56 and .71). Additionally, although the subscales of the BSPHS consisted of a 
limited number of items, the internal consistency coefficients obtained from the different samples were mostly 
above .70. According to Nunnally and Bernstein (1994), a threshold value of reliability of .70 or above for research 
and screening purposes and .90 for clinical practice is considered adequate. In this sense, the BSPHS may be used as 
an appropriate scale for research and screening purposes in college students. 

The BSPHS is a self-reported scale, in which the participants specify their levels of agreement with each item by 
marking one of the options ranging from 1 ( Strongly Disagree ) to 5 ( Strongly Agree). There is no reverse coded item 
in the scale. The total scores that can be obtained from the fear of being stigmatized by society and trust in the mental 
health professional subscales range from 4 to 20. The scores that can be obtained from difficulties in self-disclosure, 
perceived devaluation, and lack of knowledge subscales range from 3 to 15. Higher scores indicate higher obstacles 
in the related subscale and higher perceived obstacles in the related scale. In comparing the dimensions, the score an 
individual obtains from the related subscale is divided into the item number in that dimension, thus scores ranging 
from 1 to 5 may be obtained. Then, by comparing the means of these scores, it can be determined in which 
dimension an individual perceived more or fewer barriers. 

This study has some limitations that could provide opportunities to conduct new psychometric studies of the BSPHS. 
Lirstly, the validity and reliability studies were conducted with mostly non-clinical Turkish university students from 
two public universities located in urban areas of Turkey. Thus, external validity of our study is low. Luture studies 
could examine the validity and reliability of the scale on various samples, including adults and clinical samples, as 
well as examining cross-cultural validity and reliability. Secondly, although factorial, convergent, and divergent 
validity studies of the scale were conducted, the predictive validity of the scale has not been examined. Luture 
studies could also investigate to what extent the scale predicts actual psychological help seeking behavior in college 
students. 

In conclusion, considering the psychometric properties of the scale, we found initial support for the validity and 
reliability of the BSPHS for determining the barriers seeking psychological help in university students. Determining 
these barriers may help contribute to prevention and intervention efforts aimed at promoting mental health service 
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utilization and effective campaigns to reduce inhibiting factors in college students as well as to support students’ 
well-being. Since there is currently no available instrument to determine barriers to seeking psychological help in 
university students, the BSPHS may aid researchers wishing to conduct studies on this topic. Moreover, reducing the 
barriers to seeking psychological help may help some students who have mental health problems to get 
psychological help. 
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Appendix A 

The Barriers to Seeking Psychological Help Scale (Psikolojik Yardim Almaya iliskin Engeller Ol^egi) 


Instructions: There are various factors “inhibiting” peoplefrom seeking 
psychological help from a professional (psychological counselor, clinic psychologist, 
psychiatrist). Obviously, different factors inhibit people differently, but think about 
to what extent each item affect you when you need psychological help.Please, 
respond to each of the following items by blackening one number, using the 
response choices listed just below. Please try to respond to each item separately in 
your mind from each other item. Choose your answers thoughtfully, and make your 
answers as true FOR YOU as you can. Please answer every item. There are no 
"right" or "wrong" answers, so choose the most accurate answer for YOU—not what 
you think "most people" would say or do. 
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1 I have difficulty in sharing my problems with a stranger even though he is a 1 
professional.(Uzman bile olsa yabanci biriyle sorunlarimi payla^makta 
zorlamnm.) 

2 I worry about being stigmatized as “problematic” and/or “crazy” if I seek 1 
psychological help.(Psikolojik yardim alirsam “sorunlu” ve/veya “deli” olarak 
algilanmaktan fekinirim.) 

3 I don’t want to spend time seeking psychological help as it would take too 1 
long. (Psikolojik yardim aimak uzun siirebilecegi i?in zaman ayirmak 
istemem.) 

4 I don’t trust professionals to keep my issues confidential.(Psikolojik yardim 

veren uzmana, anlattiklarimi gizli tutacagi konusunda giivenmem.) | 

5 I feel ashamed to tell my problems to the professional giving psychological 1 
help. (Psikolojik yardim veren uzmana sorunlarimi anlatmaya utamnm.) 

6 I worry about what other people would think about me if I seek psychological 
help.(Psikolojik yardim alirsam insanlarm hakkimda ne/neler du§tineceklerine ^ 
ili§kin endi^elenirim.) 

7 I don’t know how to contact professionals who provide psychological help. 1 
(Psikolojik yardim veren uzmanlara nasil ula^acagimi bilmiyorum.) 

8 I refuse to give information about my private problems (sex, violence, etc.), 
even to a professional. (Uzman bile olsa birine, mahrem konulanmi (cinsellik, | 
§iddet vb.) anlatmaya (jekinirim.) 
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9 I worry about whether my friends would mock me if I seek psychological 
help.(Psikolojik yardim alirsam arkada^larimm benimle alay edeceginden j 
endi^elenirim.) 

10 1 don’t want to seek psychological help as places that provide such services are 1 
far away.(Psikolojik yardim veren yerlerin uzak olmasi sebebiyle yardim aimak 
istemem.) 

11 I would feel weak if I told my problems to a professional. (Sorunlanmi bir 1 
uzmana anlatirsam kendimi gii?suz hissederim.) 

12 1 worry that the professional wouldn’t understand me.(Psikolojik yardim veren 1 
uzmamn beni anlamayacagindan endi$elenirim.) 

13 My self-confidence might decrease if I seek psychological help. (Psikolojik 1 
yardim alirsam, kendime giivenim azalabilir.) 

14 1 don’t want to seek psychological help as it’s not accepted as “normal/natural” 

in the culture in which I grew up. (Yetifjtigim tjevrede “dogal/normal” kabul | 
edilmedigi i?in psikolojik yardim aimak istemem.) 

15 I worry that if I take psychological help once, then I would need it whenever I 
have a problem. (Bir kez psikolojik yardim alirsam, her sorun ya^adigimda ^ 
psikolojik yardima ba^vurma ihtiyaci duyarim diye endi^elenirim.) 

16 I worry about whether the professional would listen to me adequately. 
(Psikolojik yardim veren uzmamn beni yeteri kadar dinleyip dinlemeyecegi ^ 
konusunda endi^elenirim.) 

17 I worry that the professional would be insensitive to my problems, as s/he 

constantly meets people with similar problems. (Psikolojik yardim veren 
uzman, benzer sorunlan olan ki§ileri siirekli gordugiinden bana ve benim 
sorunlarima duyarsiz davramr diye endi^elenirim.) ' 
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